There’s Something About Bayes: Effective Probabilistic Programming for the Rest of Us

Bayes’ rule is a fundamental and general mechanism that composes hypotheses and data, which is arguably the purpose of many computer programs, and yet few programming languages or libraries incorporate Bayes’ rule as an abstraction. Bayes’ rule has several benefits. (1) It provides a formalism for programs to express composition of data from multiple sources to improve the program’s accuracy, e.g., combining GPS sensor data with road network map data. (2) It provides a technique for programs to implement personalization, by composing the original model with user history or preferences. (3) It expresses the potential for programs to reconsider decisions or speculate about a decision before all the data is available. This position paper explores the challenges involved in making Bayes’ rule a programming language operator in general purpose languages. We focus on average developers, who do not have deep knowledge of statistics or machine learning, and on efficiency, since developers find it challenging to reason about the hard-to-predict, potentially unbounded runtime costs of existing implementations of Bayes’ rule in inference algorithms.